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ABSTRACT 



This paper provides examples of how one can use the research 
issue and the relationships between qualitative and quantitative research as 
a frame for instructing students and judging the quality of research. The 
emphasis is on validity estimates, also called legitimization techniques, 
with attention to the idea of a qualitative-quantitative research continuum. 
It is not the technique that makes something quantitative or qualitative, but 
rather the intention of its use. The question is posed whether the research 
is testing hypotheses, helping to develop hypotheses, or describing the data. 
Criteria have been developed by Y. Lincoln and E. Guba (1985) for the 
validity of qualitative research so that it can be evaluated in a way similar 
to the more common evaluation of quantitative research. In fact, much of all 
research is a blend of quantitative and qualitative approaches. The more 
aware researchers are of this fact, the more likely it is that standards of 
good and effective research can be established. (Contains 21 references.) 
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A Conceptualization of Mixed Methods: A Need for 
Inductive/Deductive Approach to Conducting Research* 

Isadore Newman, Ph.D. 

The University of Akron 

Introduction: The purpose of this paper is to give examples of how one 

can use the research issues and interrelationships between qualitative or 
quantitative research as a frame for instructing students and judging the 
quality of research. (The intent is not to present an in depth discussion 
of the two underlying epistemologies.) Cook & Reichardt (1 979) pre- 
date our work, and like us suggest that the researcher’s method can be 
separated from his or her philosophical perception of the world. In 1 980, 
Patton presented a diagram he called a mixed paradigms. His book 
acknowledges a qualitative-quantitative continuum, however he only 
addressed qualitative methods and did not emphasize ways to critique 
research. Creswell (1 994) also discussed qualitative and quantitative 
approaches, but did not discuss how to critique research. The emphasis of 
our approach is on validity estimates, also called legitimization 
techniques, which allows one to estimate and improve the validity of their 
research, whether it is called qualitative or quantitative. And finally, the 
emphasis is on the notion of a qualitative-quantitative research 
continuum, as opposed to a dichotomy, which is more consistent with a 
coherent philosophy of science. 



*Presented at the American Educational Research Association annual 
meeting at New Orleans, LA, April 2000. 
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Quantitative research from a positivistic approach 
frequently assumes that hypotheses are derived from some 
theory or hypothetical construct, and are therefore deductive 
in nature and are sometimes referred to as theory testing. A 
summary of this concept may be conceptualized as going from: 

Theory -> General Hypotheses -> Specific Hypotheses -> Data 

Collection -> Data Analysis -> Results -> 

Conclusion -> Theory Confirmation/Revision. 

On the other hand, a qualitative or constructivist approach to 
research may be considered inductive and the purpose is not to test 
theory but to generate theory and explanations of phenomena. This can 
be conceptualized as starting with: 

Data Collections Data Analysiss Conclusions^ Development of 

Hypothesess leading to Theory Development. 

Obviously, these methods are most appropriate for answering 
different questions. For example, if your questions are to test differences, 
to generalize (to infer from the sample to a population), or to test theory, 
you are more likely to be doing quantitative research and the research 
methodology tends to be pre-experimental, quasi-experimental, true- 
experimental or expost-facto research designs. This would include many 
path- analytic studies. Some of the assumptions of these methods are 
that data can be collected value free, there is independence of 
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measurements, and an objective reality exists. Therefore, one can make 
generalizations. 

If the intent is to describe, to uncover deep meaning, to explain, or 
to build theory, you are more likely to be interested in conducting 
qualitative research. Generally, research methods associated with these 
types of questions tend to be case studies, field studies, historiographic, 
ethnographic, anthropological, document, and phenomenological. These 
methods assume that all data is value laden, context embedded, subjects 
and objects are dependent, everything is relative, and there is no objective 
reality. Therefore, neutrality of data is not possible and one cannot 
generalize. 

Quantitative research starts with some theoretical or hypothetical 
construct from which hypotheses are derived. The theory is an outgrowth 
of qualitative conceptualizations. The point is that one can not do 
qualitative research that is independent of some previously developed 
qualitative concepts. While quantitative research, which is based on 
positivistic philosophy, strives to be objective, it contains many subjective 
aspects. For example, the selection of a subjective alpha level, 
directionality of hypotheses, and the interpretation and implication of data 
all have subjective qualities. 

Confusion between “Qualitative and Quantitative Methods” and 
“Qualitative and Quantitative Research.” 

There is a frequently help misconception that quantitative research 
uses numbers and qualitative research is narrative. This is a misleading 
over simplification. As stated above, qualitative research starts with data, 
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analyzes the data, makes conclusions, and ends with developing 
hypotheses and/or theory. Exploratory factor analysis, which is a 
sophisticated statistical technique, starts with the data, analyzes the data, 
interprets the data (conclusions) and infers the underlying constructs that 
the set of data is measuring (hypothesized or theoretical underlying 
dimensions). Therefore, by definition, Exploratory Factor Analysis is 
isomorphic with the intent and procedures of qualitative analysis. On the 
other hand, Confirmatory Factor Analysis, which is virtually statistically 
identical, actually tests to determine if the derived factors fit the 
hypothesized factors. Therefore, Confirmatory Factory Analysis begins 
with the hypothesized factors, collects data, analyzes the data, makes 
conclusions about the match between the observed and hypothesized 
factor. This is isomorphic with the intent and procedures of quantitative 

research. 

One can also make the same argument for the use of interview data, 
which can legitimately be used as a data source in qualitative or 
quantitative research. The point tha t is being made is that i t i s not the 
tprhnigiie that makes somethin g quantitative or Qualitative , but It I S tb fi 
jntpnt of its uses. Is it testing h ypotheses or is it he l ping tQ develop 
hypotheses or describe the .data . 

Differences Between Mixed Philosophies versus Mixed Methods: 

Logically, one can not simultaneously hold two philosophical 
positions that have contradictory assumptions. This does not mean one 
can not simultaneously use mixed methods for any particular study. 
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Gueulette, Newgent, & Newman (2000) reviewed the definitions of 
qualitative and quantitative research presented by Soltis (1990), Miles 
and Huberman (1984), Eisner & Peshkin (1990), Gay (1992), Rubinowitz 
& Weseen (1997), Hoshmond (1990), Newman & Benz (1998) and 
others. Based upon the Gueulette et al review, they developed criteria for 
classifying research as being quantitative or qualitative. These criteria 
were then used to evaluate 379 references that had been identified by 
the authors, journals or index as qualitative research. Out of this sample 
of 379 references, 44.06% (167) were identified as being a blend of 
qualitative and quantitative methods, even though they were originally 
classified only as qualitative research. It becomes evident from their work 
that the literature is very unclear about what they identify as qualitative 
or quantitative research. 

All research should theoretically or logically be able to be classified 
as being well done or poorly done. Therefore, a need for standards of 
quality is necessary. In quantitative research, generally accepted 
standards for evaluating the quality of research have been presented by 
Campbell and Stanley (1 963) and Campbell & Cook, (1 979). These 
standards relate to the internal and external validity of the research 
design. For qualitative research, there tends to be less agreement as to 
what is good research, to the extent that some prominent qualitative 
researchers (Schwandt, 1 990) would say that standards are inappropriate. 
Others might say that good qualitative research is less dependent on the 
methods and more related to the telling of the story. 

Goetz and LeCompte (1984), Newman & Newman (1994), Newman 
& Benz (1 998), Guba & Lincoln (1 985), Lincoln (1 995), Tashakkori & 
Teddlie (1 998) and Denzin (1 994) have supported the need for standards 
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for qualitative research. A list, which should not be considered exhaustive, 
has been suggested by Lincoln and Guba (1 985), and added to by others. 
This list provides a foundation for developing standards to evaluate the 
quality of qualitative research in a way that is similar to the Campbell and 

Stanley criteria for quantitative research. 

The Campbell and Stanley criteria for quantitative research includes: 
Threats to Internal Validity: History, Maturation, Testing Effects, 
Statistical Regression, Experimental Mortality, Selection Bias, 
Instrumentation (Content, Expert Judge, Face, Concurrent, Predictive and 
Construct Validity, Test-Retest Reliability, Equivalent Forms, Internal 
Consistency). 

Threats to External Validity: Interaction of selection and treatment, 
Interaction of setting and treatment; Interaction of history and treatment; 
Reactive arrangements, etc. 

For qualitative research there are 1 5 criteria suggested mainly by 

Lincoln and Guba (1985). These are: 

Neutrality, Prolonged Engagement, Persistent Observation, Peer De- 
briefing, Triangulation, Member Checking, Structural Relationships, 
Theoretical Sampling, Audit Trail, Negative Case Analysis, Thick 
Descriptors, Referential Adequacy, Overlapped Methods, Step-wise 
Replication, and Reflective Journal Writing. 

There are many well understood available examples demonstrating 
the use of Campbell and Stanley criteria for evaluating quantitative 
research but there are less available examples demonstrating the 
application of the above considerations to qualitative research. Therefore, 
what follows is an example of how one can use qualitative criteria in a 





7 



similar manner to help evaluate the quality of qualitative research. The 
example comes from an article critique by Newman & Benz (1 998) of a 
case study of the aftermath of a student’s suicide at a middle school. It 
concludes with a heading called Reflections, which is a section that asks 
the person critiquing the article to suggest what could or should be done 
quantitatively, following the qualitative analysis. The critique addresses 
the authors use or lack of use of the several qualitative methods listed. 

Neutrality: Since there was one observer and there was no 

attempt to control for subjective perceptions and personal biases, it is our 
estimate that this article was weak in the neutrality criterion. 

Prolonged On-Site Engagement: The researchers met this 
criterion since the author /researcher was a counselor on-site prior to the 
suicide and for at least 1 2 weeks following the suicide. 

Persistent Observation: The author infers that there was an 
increase in suicide ideation after the suicide incident. However, there were 
no pre-assessment of suicide ideation and no attempt to collect data on 
which these inferences could be made. Therefore, it is assumed that this 
criterion is weak or was not met. 

Peer De-briefing: According to the data presented in the article, 
there appeared to be no attempt to confirm perceptions and 
interpretations with other psychologists, counselors, social workers, or any 
other mental health workers. 

Triangulation: It did not appear that there was any attempt to 

tap information from other sources such as mental health records, school 
records, parents, etc. 
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Member Checking: Based upon interviews with students the 
counselor identified emerging themes that she did not proceed to check in 
any consistent fashion. 

Structural Relationships: There was a minor attempt to explain 
some of the behaviors of students from a Gestalt theoretical perspective. 
However, the author did not interweave different data to develop the 

theoretical (structural) conceptualization. 

Theoretical Sampling: There was no attempt at soft hypotheses 
testing. That is no hypotheses were developed based upon the existing 
data, and no additional data was collected to determine if hypotheses 
were supported. 

Leaving an Audit Trail: It is possible that the short format of the 
journal article did not allow for presentation of all data sources. Only the 
author’s subjective perceptions were presented. 

Generalizability: Appropriately there was no attempt in this article to 

generalize. The author was poor in providing deep descriptors, that is 
detailed descriptions of the subjects, situation, and the culture of the 
school. There also appeared to be no attempt at negative case 
analysis, that is there was no attempt to explain outliers from their 
particular perspective. 

Truth Value, Credibility, Confidence In the Research: The stated 

purpose of the article was to demonstrate the effectiveness of a Gestalt 
therapeutic approach in helping student suffering from trauma do to a 
peer’s suicide. Based upon the above criteria, it would be difficult to 
conclude anything about the relative effectiveness of the Gestalt 
approach. 



o 

ERIC 



10J 



9 



Reflections: Using the qualitative/quantitative interactive continuum as 

a guide, one could use the themes that emerged from this case study in 
future research to evaluate the effectiveness of treatments by randomly 
assigning students to different treatment groups such as Gestalt, 
behavioral, psychoanalytic, etc. This quantitative procedure would then 
build on the heuristic and descriptive qualitative findings to enhance the 
researcher’s ability to estimate the effectiveness of a particular 
treatment. 

Conclusion: As stated in Newman and Benz (1 998) and supported by 

the review of the literature presented by Gueulette, Newgent, & Newman 
(2000) much of all research tends to be blends (mixed models, 
qualitative/quantitative). Researchers needs to recognize that to build 
knowledge for any discipline or field one must be willing to use both 
quantitative and qualitative methods as appropriate. As Newman and 
Benz indicated, each approach can provide insights on which the other 

methods can build (reflectiveness). 

It is impossible for me to think of doing a quantitative study without 
making a significant number of qualitative judgements. To the extent that 
researchers are aware of this, the more likely it is that standards of good 
and effective research can be established. This acceptance of the need 
for both conceptualizations will decrease wasted time that is spent in 
arguing about a dichotomy that is neither fruitful or effective. The 
emphases should be on identifying and clearly stating the research 
questions of interest and the most appropriate methods for answering 
1 those questions. This requires researchers to know the strengths and 
weaknesses of each. The approach suggested in this paper will lead to the 
improved research and improved training of future researchers. 
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